Summative Evaluation

Summative evaluations are more specific than formative evaluations. As the name implies, they provide a summative assessment of the programme. They attempt to determine the effect of a programme’s actions and activities in reaching its goals. Summative evaluations examine the effects or outcomes of a programme/intervention by ascertaining whether the outcomes are following the desired objective. A summative evaluation may also be sub-divided into Impact Evaluation and Outcome Evaluation.

Outcome Evaluation

The objective of outcome evaluation is to assess the impact of a programme’s operational and service delivery on its desired results. It is a technique that determines the long-term outcomes of a programme and helps in identifying if it has generated the anticipated outcomes and the factors contributing to obtaining those objectives. Additionally, examining outcomes focuses on unwanted consequences or variables that may prohibit programmes from attaining the targeted goals. Implementers, funders, and other stakeholders may evaluate outcomes to assess a programme’s efficacy by comparing various strategies to achieve the same objectives.

In this context, it is essential to acknowledge that the intended objectives may be more challenging to attain in actual conditions. While short-term results might occur throughout programme implementation, long-term consequences often do not arise until after programme implementation. Therefore, dividing result categories into short, medium, and long-term groups is vital to evaluate the different outcomes throughout the implementation period. It is also crucial to construct a specific timetable of results that may identify the timeframe for each outcome hierarchy—short, medium, or long-term.

Short, medium, and long-term outcomes are typically determined using quantitative research design and strategies. However, in recent years, there has been an emphasis on using mixed methods and designs to thoroughly examine outcomes to ascertain the results and identify the aspects or tactics (both formal and process) that lead to the results.

Impact evaluation

Impact evaluation is a sort of evaluation that attempts to establish the relationship between the intervention and the programme’s outcome, identifying casual relationships between programme intervention and effect.

In contrast to other types of assessment, the primary objective of impact evaluation is to establish causal linkages and attribute changes to the programme intervention. The best method to address the problem of attribution is to design a counterfactual, which involves determining what the result would have been if the intervention had not occurred. There are several ways to create a counterfactual. Depending on the ways selected to create a counterfactual or control group, Impact evaluation can be further categorized into broader categories:

a) Experimental,

b) Quasi-Experimental, and

c) Non-Experimental Methods.

It is vital to remember that a control group is used when an experimental design is employed. Typically, the phrase “comparison group” refers to a quasi-experimental strategy.

Some of the popular methods of Impact evaluation are :

Experimental Design

• Randomised Control Trial (RCT)

Quasi-Experimental Design

• Difference-in Difference Design
• Propensity Score Matching
• Regression Discontinuity
• Interrupted Time Series
• Instrumental Variable/Randomised Promotions

Non-experimental

• Synthetical Control Method
• Fixed Effect/Random Effect Model in case of Longitudinal Design
• Contribution Analysis with Process Tracing

References:

ActionAid (2016). How to… Select a Methodological Approach for the Evaluation. Evaluation Technical Briefing Note, #7. ActionAid, UK. § IFRC (2011).

Project/programme Monitoring and Evaluation (M&E) Guide. International Federation of Red Cross and Red Crescent Societies. Geneva, 2011.

Rossi, Peter Henry, Mark W. Lipsey, and Howard E. Freeman. “Evaluation: A systematic approach.” Sage Publications, 2004.

Kultar Singh – Chief Executive Officer, Sambodhi